Sparse Graph Attention Networks

Graph Neural Networks (GNNs) have proved to be an effective representation learning framework for graph-structured data, and achieved state-of-the-art performance on many practical predictive tasks. Among the variants of GNNs, Attention (GATs) improve graph tasks through a dense attention mechanism. However, real-world graphs are often very large noisy, GATs prone overfitting if not regularized properly. In this paper, we propose Sparse (SGATs) that learn sparse coefficients under L0-norm regularization, learned attentions then used all GNN layers, resulting in edge-sparsified graph. By doing so, can identify noisy/task-irrelevant edges, thus perform feature aggregation most informative neighbors. Extensive experiments synthetic (assortative disassortative) benchmarks demonstrate superior SGATs. Furthermore, removed edges interpreted intuitively quantitatively. To best our knowledge, is first algorithm shows significant redundancies achieve similar (on assortative graphs) or sometimes higher disassortative performances than original graphs. Our code available at https://github.com/Yangyeeee/SGAT.